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Abstract 

This paper focuses on the estimation of low-complexity signals when they are observed 
through M uniformly quantized compressive observations. Among such signals, we consider 
1-D sparse vectors, low-rank matrices, or compressible signals that are well approximated 
by one of these two models. In this context, we prove the estimation efficiency of a vari¬ 
ant of Basis Pursuit Denoise, called Consistent Basis Pursuit (CoBP), enforcing consistency 
between the observations and the re-observed estimate, while promoting its low-complexity 
nature. We show that the reconstruction error of CoBP decays like M -1 / 4 when all param¬ 
eters but M are fixed. Our proof is connected to recent bounds on the proximity of vectors 
or matrices when (i) those belong to a set of small intrinsic “dimension”, as measured by 
the Gaussian mean width, and (ii) they share the same quantized (dithered) random pro¬ 
jections. By solving CoBP with a proximal algorithm, we provide some extensive numerical 
observations that confirm the theoretical bound as M is increased, displaying even faster 
error decay than predicted. The same phenomenon is observed in the special, yet important 
case of 1-bit CS. 

Keywords: Quantized compressed sensing, quantization, consistency, error decay, low-rank, 
sparsity. 


1 Introduction 

The theory of Compressed Sensing (CS) shows that many signals of interest can be recon¬ 
structed from a few linear, and typically random, observations umm- Interestingly, this 
reconstruction is made possible if the number of observations (or measurements) is adjusted to 
the intrinsic complexity of the signal, e.g., its sparsity for vectors or its low-rankness for matri¬ 
ces. Thus, this principle is a generalization of the Shannon-Nyquist sampling theorem, where 
the sampling rate is set by the bandwidth of the signal. However, a significant aspect of CS 
systems is the effect of quantization on the acquired observations, in particular for the purpose 
of compression and transmission mmmmmmm- This quantization is a non-linear 
transformation that both distorts the CS observations and increases, especially at low bit rates, 
the reconstruction error of CS reconstruction procedures. 

This work focuses on minimizing the impact of (scalar) quantization during the reconstruc¬ 
tion of a signal from its quantized compressive observations. While more efficient quantization 
procedures exist in the literature (e.g., EA (2X|, universal [3], binned [23 [30] , vector (29, [30] 
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or analysis-by-synthesis quantizations m ), scalar quantization remains appealing for its im¬ 
plementation simplicity in most electronic devices, and for its robustness against measurement 
lost. 

Conversely to other attempts, which consider quantization distortion as additive Gaussian 
measurement noise [8] and promote a Euclidean (£ 2 ) fidelity with the signal observations as in the 
Basis Pursuit Denoise (BPDN) program, better signal reconstruction methods are reached by 
forcing consistency between the re-observed signal estimate and the quantized observations m 
[23135]. 

We show here that a consistent version of the basis pursuit program Em, coined CoBP, 
provides better signal estimates at large M than those obtained by BPDN. When reconstructing 
sparse or compressible signals, CoBP is similar, up to an additional normalization constraint, to 
former methods proposed in [13H151 [22 ]. We prove the efficiency of CoBP from recent results on 
the proximity of signals when those are taken in a set 1C C of small “dimension”, he., with 
small Gaussian width w(JC) [1] 110] . and when their quantized random projections are consistent 
ES [25]. In particular, we show that for sub-Gaussian sensing matrices, the ^-reconstruction 
error of CoBP decays as v^W/m 1 / 4 , with an additional constant error bias arising in the case 
of non-Gaussian sensing matrices. This contrasts with BPDN, whose reconstruction error is 
only guaranteed to saturate when M increases. 

The rest of this paper is structured as follows. Sec. [2] introduces the problem by explaining 
the low-complexity signal space, our Quantized Compressed Sensing (QCS) model and the 
BPDN reconstruction procedure as generally used in QCS. Sec. [3] reviews important results on 
the proximity of consistent vectors; in Sec. [4] we introduce and analyze CoBP. Finally, Sec. [5] 
demonstrates experimentally the capabilities of this method in QCS of signals and matrices, 
before concluding. 

Conventions: Vectors and matrices are associated to bold symbols. The probability of an 
event X is P(A). The identity matrix is 1 £> G M. DxD ( D G N), [D] := {1, ••• , D} and |<S| 
is the cardinality of S C [D]. The ^ p -norm of u is ||«|| p and the unit £ p -ba!A is B^ = {x G 
R. n : ||x|| p ^ 1}, with B^ := B^. Assuming N = n 2 is a square number, for a matrix 
U = ( ui , • • • ,u n ) G R nxn with vectorization vec(U ) := (uf, ■ ■ ■ ,u^) T G R N , rank(£7), ||C7||, 
||C/||* and \\U\\ F := tr (U T U) 1 / 2 = || vec ([/)||2 denote its rank, operator norm, nuclear norm 
and its Frobenius norm, respectively. We will often assimilate matrices in M nxn with their 
vectorization in W. N , e.g., identifying {U G M nxn : ||D"||_f ^ 1} with B^. Finally, we write / < g 
or / = O(g) if / ^ eg for c > 0, and similarly for f > g and / = Q(g). 


2 Quantized Compressed Sensing of Low-Complexity Signals 

2.1 Low-complexity Signal Model 

This work focuses on the sensing of signals belonging to a low-complexity set /C C . A typical 
example is the set of A'-sparse vectors 1C = T*k := {u G : ||«x||o := |supp u\ ^ K}, as well as 
the set of rank-r matrices C r := {U G M nxn ~ W N : rank(C7) ^ r}. 

As in m, we assume that the (bounded) convex hull 1C := con v(/C H B^) of 1C is associated 
to the definition of an appropriate atomic norn^] || ■ ||j) such that 

1C = fC s := {u G : ||u||t| ^ s, ||tt ||2 ^ 1}, (1) 

1 If K, is convex and centrally symmetric around the origin, || ■ || j can always be defined by the gauge of tC (see 

m for details). 
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for some s > 0. For instance, for compressible signals in T,k, ||-||jj = ||-||i and s = \[K. while 
for matrices in C r , |j • ||g = || • ||* for s = yfr [32]. 

The “low-complexity” nature of these sets stems from their small Gaussian mean width 

w(K) := E sup \g T u\, g ~ AA(0, In)- 

u£K 

For instance, w(T,k) 2 = rc(Sx) 2 < K log n /k and w(C r ) 2 = w(C r ) 2 ^ 4nr [UUP]. The quantity 
w(IC), also called Gaussian complexity, has been recognized as central, e.g., for random pro¬ 
cesses characterization [36], high-dimensional statistics and inverse problem solving mm or 
classification in randomly projected domains [2]. As explained below, w(IC) also determines the 
minimal number of measurements for CS of signals in K. [10] . 

2.2 Quantized Compressed Sensing 

Given a certain quantization resolution 5 > 0, we focus on the impact of a uniform (midrise) 
quantizer Q(t ) := 5([|J + |) £ Z ,5 := 5(Z± ^), applied componentwise, in the quantized sensing 
model 

g = A(xo):=Q(fx 0 + ^)6Zf, (2) 

where $ € R MxN is a random sensing matrix and £ ~ U M ([—5/2,6/2]) (i.e ., C ~iid U([—5/2, 5/2]) 
for i £ [M]) is a uniform dithering] This random dithering is known at the signal reconstruc¬ 
tion and stabilizes the action of Q [31 I2U1I23] . By slightly abusing the notation, when ([2]) senses 
an element Xq of a matrix set in M nXTl , xq = vec(Xo) amounts to the A-length vectorization 
of this element, assuming N = n 2 . 

As often the case in CS, we consider that $ is a sub-Gaussian random matrix, i.e., its entries 
are distributed as ~iid ip with ip a symmetric, zero-mean and unit-variance sub-Gaussian 
random variable (r.v.), having finite sub-Gaussian norm 

Milk := su Pp^lP~ 1/2 ( IE l^| P ) 1/p < oo. 

For such a r.v. of sub-Gaussian norm a > 0, we have in fact P[|^| > t\ < exp (—ct 2 /a 2 ) for any 
t > 0. Examples of such r.v.’s are Gaussian, uniform, bounded or Bernoulli distributed r.v.’s. 
Below, we write ip ~ A4g,«(0,1), and the shorthand rsj i¥ (0,1) for the associated M x N 
matrix, to specify that ip is a sub-Gaussian r.v. of norm a. 

In the absence of quantization, if M > w(IC) 2 , with high probability, any xq £ K. can be 
reconstructed from sub-Gaussian observations <ha;o using convex optimization programs such 
as Basis Pursuit m • Therefore, the minimal number of measurements needed for reconstruct¬ 
ing AT-sparse or compressible signals in grows like Klog N /K, and like nr for rank-r and 
compressible n x n matrices mm- 

l-bit Quantization Regime: The exponentially decaying tail bounds of the sub-Gaussian 
entries of $ show that a suitable value of 5 can essentially turn ([2]) into a 1-bit CS model when 
K, is bounded [U, 26, 33]. Indeed, from the definition of Q and assuming || a?o || 2 = 1, fori £ [M], 
P[gi ^ {±5/2}] = P[\ipfx + £j| ^ 5] = pQ ^ 2exp(— ^5 2 ), with po = 0.0027 for 5 = 3. Our study 
holds in such a regime with the interesting advantage of allowing the estimation of the signal 
norm, as opposed to the 1-bit CS model sign (<ha:o) m E2]. This is due to the pre-quantization 
dithering in ([ 2 ]). Interestingly, combining the sign operator with prequantization thresholds in 
1-bit CS also removes this signal norm uncertainty [28] . 

2 As in [25], our results remain valid if £ ~ U M ([t, t + 5]) for any t £ R. 
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2.3 Basis Pursuit Denoise 

The first method used to estimate xq from q in ([ 2 ]) was considering quantization as an additive 
noise of bounded power under high resolution assumption (HRA), i.e., 6 <C ||*o||2 [6] - In ([2]), the 
impact of the dithering provides q = (Qx + £) + n with n := q — (Qx + £) r\_/ U M ([-<5/2, 5/2]). 
Therefore, ||n|| 2 ^ e 2 = holds with high probability for small k (e.g., k = 2) [201122] . 

In such a case, the general BPDN program, 

■®BPDN := &rgmin ||u||jj s.t. ||3>u + | — q|| 2 ^ e (BPDN) 

u£R n 

can be solved for estimating xq. When ^/-/m satisfies the restricted isometry property (RIP) 
and when 1C is the set of sparse signals, then, setting || ■ ||y = || • 111 , [2D [22] show that 

||®* — cc 0 || 2 = 0(*/Vm) = 0(5). 

A similar result holds in the case of QCS of low-rank matrices using a Lasso reconstruction that 
minimizes a Lagrangian formulation of BPDN [T]. Notice that a variant of BPDN, called Basis 
Pursuit DeQuantizer of moment p (BPDQ [22]), replaces the f^-norm of the BPDN constraint 
by an £ p -norm (2 ^ p < oo). Its error decays like 0(5 / y/\ogM) [5j. 


3 Proximity of Consistent Vectors 

This section summarizes a recent study showing that the proximity of vectors of a subset 
1C C R. N with small Gaussian mean width can be bounded provided they share the same image 
through the random mapping A, i.e., if they are consistent [25] , As will be clear in Sec. |4j this 
property is the key for characterizing the behavior of CoBP. 

This proximity is impacted by the level of anisotropy of the sub-Gaussian rows composing 
$ rs_/ ACho.i) m, as measured by the smallest K sg ^ 0 such that, for <p r\j Klai 0,1), 
<7 ~ J\f N ( 0 , 1) and all u E R N , 

/o + °° | p (l(<Aw)| >t)~ P(|(gf,u)| ^ t) | dt ^ K sg 11tt11oo• (3) 

For Gaussian (isotropic) random vectors « ag = 0, while for sub-Gaussian (p rs_/ Klai 0-1), 
K sg ^ 9\/27a 3 , with a ^ 1 for Bernoulli r.v.’s [25] , 

As clarified in Prop. [D when the mapping A integrates a non-Gaussian, but sub-Gaussian 
sensing matrix <!>, the proximity of consistent elements x,y in 1C is guaranteed when x — y is 
not “too sparse”, i.e., when it belongs to 

Zk 0 ■= {ueR N : AolMlL < IMllD 

for Kq large enough compared to K ag . For instance, a JF-sparse vector u E T,k '■= {u : ||v||o := 
|suppu| ^ K} cannot belong to Hk 0 for K$ > K as then ||w||| ^ K ||tx||^. 

Proposition 1 (Consistency width [2D). Given a quantization resolution 5 > 0, e G (0,1), a 
sub-Gaussian distribution AT sg ^ a (0 ,1) respecting © for 0 ^ At ag < oo, and 1C C M N a bounded 
subset ofR N , there exist some values C,c> 0 depending only on a and such that, if 

M>C w(K,) 2 , (4) 

then, for $ ~ (0, 1), £ ~ U M (\—5/2, 5/2]) and \Z~Kq ^ 16K ag , with probability exceeding 

1 — 2 exp(— ceM/(l + 5)), we have for all x,y G 1C 

x - y G Z Ko , A(x) = A(y) =>- I® - y|| 2 ^ e, (5) 
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with A defined in © Moreover, for any orthonormal basis \E r E K NxN , if 1C = (4/Sa') 
then Q simplifies to 

M^C'^Kl og(^(^) 3/2 ), ( 6 ) 

for some C' > 0 depending only on a. 

We remark that for Gaussian sensing matrices, the “antisparse” condition on x — y (and 
on /t 0 ) vanishes since re ag = 0. This provides, in the special case of the sparse signal set, a 
proximity bound in ([5]) formerly established in 24] . 


4 Consistent Basis Pursuit 

The previous sections allow us now to define a suitable reconstruction procedure for estimating 
any signal xq E KL s (for some s > 0 in ([!])) observed through the model ([2]), e.g., for reconstruct¬ 
ing compressible signals or matrices belonging to or C r , respectively. We split the study 
according to the nature of the sensing matrix. 


4.1 Gaussian Sensing Matrix 

When $ is Gaussian, i.e., K sg = 0, we propose to estimate xq with the following program coined 
Consistent Basis Pursuit, 

x* := argmin ||u||n s.t. A(u) = A(x o), u E B^. (CoBP) 

ueR N 

This is a convex optimization as the first constraint is equivalent to ||4>« + £ — A(*o)||oo ^ 
6/2 [22]. The proximity of x* to xq is then guaranteed by Prop. [I] 

Proposition 2. If A respects ([5]) for all x,y E K s and = 0, then for all x$ E 1C, the 
estimate x* obtained by CoBP from q = A(x o) satisfies ||xq — **||2 ^ e. 


Proof. Since *o E IC s is a feasible vector of the CoBP constraints, we necessarily have ||aP||{| ^ 

||a?oHtt ^ s ■ By definition of CoBP, x* E so that x* E K, s . The result follows from ([5]) with 
x = xq and y = x*. □ 

Prop. [ 2 ] assumes that Kq = 0 in ([5]) . This holds if K ag = 0, e.g., if 4> ~ Af MxN (0,1). 
Therefore, combining the conditions of Prop. [I] with this last proposition, we get the following 
corollary by saturating Q with respect to M. 

Corollary 1. Given some universal constant c > 0, with probability exceeding 1— 2exp(— cM 3 ^ 4 /\fid) 
over the draw of 4> ~ Af MxN (0, 1) and £ ~ U AI ([—5/2, 6/2]), for every xq E K, s , the estimate 
x* obtained by CoBP from q = A(xq) satisfies 

ll*0-*1l2 = O(iM(=!§£ ) 1/4 ), 

i.e., ||cco — **11 2 = 0(M -1 / 4 ) if only M varies. 


At first sight, the error decay of CoBP in M^ 1 / 4 could seem slow. However, as mentioned in 
Sec. 2.3, the best known error decay for BPDN under the sensing model (|2j) is 0(5) [2Tj, which 
does not decay with M. The same constant bound was found for a variant of CoBP without 
the ball constraint m- 
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4.2 Non-Gaussian Sensing Matrix 

For non-Gaussian 4>, K sg 7 ^ 0 in general. In order to reach a meaningful estimate of xq E /C s , we 
further assume that ||so||oo ^ A, for some A > 0 . As will be clear, this allows us to characterize 
the sparse nature of xq — x* when x* is an estimate of xq produced by the modified program: 

x* := argmin ||n||jj s.t. / ^ ^ ^ °2’ (CoBPa) 

u % r n 11 \uen N nxu^ 

with CoBPa = CoBP as soon as A > 1 since M N C B^. 

Proposition 3. If A respects ([5]) for all x,y E IC S and any Kq ^ (16k ss ) 2 , then for any 
xq E KL s n AB^ ; the solution obtained by CoBP\ from q = A(x 0 ) respects 

||*o ~ x * ||2 ^ e + 2A\/ATo- 

Proof. As for the proof of Prop. [ 2 J xq E KL s implies that x* E fC s . If ||xo — x *||2 ^ \/-^o||*o — 
x*||oo, then, since xo,x* E AB^, |®o — 1 1 2 ^ 2\t/Kq. Otherwise, we have xq — x* E 2r- 0 . In 

this case, since © is assumed satisfied for all pairs of vectors of /C s , we have ||®o — x *\\2 ^ e, 
which concludes the proof. □ 

Taking A'q = |~(16K sg ) 2 ~|, this corollary is easily established. 
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Corollary 2. Given some universal constant c > 0, with probability exceeding 1 — 2 exp(— c M ^ ) 
over the draw of $ ~ A/gg^ jV (0,1) and £ ~ U M {[—6/2, 5/2]), the CoBP\ estimate x* of any 
xq e JC s n AB^ satisfies 

ll*o -**||2 = 0(^(^%^) 1/4 + K ag A), (7) 

i.e. ; ||cco — x* || 2 = 0(M -1 / 3 4 + ft ag A) if only M varies. 

Loosely speaking, Cor. [ 2 ] shows that the reconstruction error is not guaranteed to decay 
below a certain level fixed by K sg ||*o||oo- A similar behavior was already observed in the case 
of 1 -bit CS with non-Gaussian measurements [Ij. 


5 Experiments 


In this section we run several numerical simulations in order to assess the experimental benefit 
of CoBP compared to BPDN in various QCS settings. As CoBP is a convex optimization prob¬ 
lem containing non-smooth convex functions, we solv^Jit with the versatile Parallel Proximal 
Algorithm (PPXA) [T2], this one being efficiently implemented in the UNLocBoX toolbox |31| . 
We refer the reader to [18] for an example application of PPXA in the solution of low-rank 
matrix recovery. 


For our experiments, three different sensing contexts are tested: the two first ones consider 
QCS of sparse signals (for Gaussian or Bernoulli sensing matrices), while the last one focuses 
on QCS of rank -1 matrices. In all cases, the quantization resolution is fixed by <5 = 6 x 2 1 ~ B 
with B E [4], As explained in Sec. 2.2 each q t can then be essentially coded with B bits, e.g ., 
if B = 1, E|{l : qi ^ {±<f/2}}| ^ 0.0027M. Some of our results are compared to those of BPDN 
with e set as in Sec. |2.3[ The constraint “u E B^” is also added to BPDN for reaching fair 
comparisons with CoBFpJ 


5.1 Gaussian QCS of sparse signals 


In this experiment, we set N = 2048, K = 16, B = 3 and M/K E [8,128], i.e., well after 
the phase transition (here around M/K ~ 6 ) where sparse signal reconstruction from noisy CS 
measurements is guaranteed [ 8 j. For each value of M, 20 different Gaussian sensing matrices, 
dithering realizations and unit-norm A'-sparse signals were randomly generated. Each signal 
*o has its A'-length support selected uniformly at random in [IV], with non-zero components 
drawn as A7(0,1) before normalization. The reconstruction error decay averaged over these 
20 trials is shown for BPDN, BPDQ with p = 4 (see Sec. 2.3) and CoBP in Fig. |??[left) in a 
log 2 / log 2 plot. For indication, a linear fitting over the last 4 values of log 2 M/K provides slopes 
of value —0.31, —0.33 and —0.95 for BPDN, BPDQ and CoBP, respectively. As already observed 
experimentally in other works forcing tight or approximate consistency in signal reconstruction 
this clearly highlights the advantage of consistent signal reconstruction when 
M/K is large. Moreover, CoBP approaches an error decay of M _1 similar to the distance decay 
of consistent A'-sparse vectors when ([ 6 ]) is saturated, i.e., better than the of Cor.[lJ 


3 Free matlab code: http://sites.uclouvain.be/ispgroup/index.php/Softwares 

4 The ratio of computational times between CoBP and BPDN is about 1.3. 
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5.2 Bernoulli vs Gaussian QCS 


This second experiment stresses the impact of the sub-Gaussian nature of the sensing matrix 
over the CoBP reconstruction error. We focus on the case of Gaussian QCS (3> M MxN (0,l )) 
and Bernoulli QCS (he., equals ±1 with probability 1/2) when observing A'-sparse signals 
for K growing and M/K constant. In particular, we set N = 1024, B = 4, K 6 [1,64] and 
M/K = 16. For each value of AT, 20 different sensing matrices, dithering realizations and 
unit-norm A'-sparse signals are generated as in the first experiment. CoBP and CoBP,\ are 
compared (with an oracle assisted A := ||xo||oo)- Comparing the error bounds for Gaussian and 
sub-Gaussian QCS in Cor. [I] and in Cor.[2j respectively, we expect that at low K and for M/K 
constant, Bernoulli QCS reaches worst reconstruction error than Gaussian QCS , as then the 
bias K ag A = K sg ||£Co ||oo — k bs /VK can be high. This is indeed observed in Fig. ?? [middle) with 
a clear gap between Bernoulli and Gaussian QCS performances when K ^ 16. CoBP^ does 
lead to clear improvements over CoBP. 


5.3 Gaussian QCS of rank-1 matrices 

We reconstruct here rank-1 matrices in M 32x32 (he., N = 1024 and n = 32) from the Gaussian 
QCS model ([2]) with B € {1,2}. Both CoBP and BPDN are solved with ||-||n = ||-||*. The 
intrinsic complexity of such rank-1 matrices is 63 < P := 64. For each value of the oversampling 
ratio M/P £ [4,32], we generate 20 different Gaussian sensing matrices, dithering realizations 
and rank-1 matrices according to xq = vec(Wo) and Xo = with v ~ J\f n (0,1 ). As for 

the first experiment on A'-sparse signals, CoBP reaches a faster reconstruction error decay than 
BPDN. At B = 2, an indicative linear fitting over the last 4 values of M/K provides estimated 
decay exponents for CoBP and BPDN of —0.85 and —0.33, respectively. 


6 Conclusion 

In the context of QCS of signals with low-complexity ( e.g ., sparse signals, low-rank matrices), 
we show that the consistent reconstruction method CoBP has an estimation error decaying as 
M^ 1 / 3 4 , he., faster than the one of BPDN. This is confirmed numerically on several settings 
with even faster effective decaying rate at quantization resolution as low as one bit per mea¬ 
surement. As observed initially in 1-bit CS [1], QCS performances for general sub-Gaussian 
sensing matrices are also impacted when the sensed signal is “too sparse”. Finally, to the best 
of our knowledge, we provided the first theoretical analysis of CoBP in the case of low-rank 
matrix reconstruction from QCS observations. 
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